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^ [ Abstract. We introduce a notion of ^-Volterra quadratic stochastic operator defined 
on (m — l)-dimensional simplex, where £ G {0,1,..., m}. The £- Volterra operator is a 
Volterra operator iff £ = m. We study structure of the set of all Volterra operators and 
describe their several fixed and periodic points. For m = 2 and 3 we describe behavior of 
trajectories of (m — l)-Volterra operators. The paper also contains many remarks with 
comparisons of £- Volterra operators and Volterra ones. 
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c4 ■ 1 Introduction 

; 

In biology a quadratic stochastic operator (QSO) has meaning of a population evolution 
> . operator (see [15] ; [IB] . [17]). which arises as follows. Consider a population consisting of 
^ ■ m species. Let x° = ■■■ 1 x° m ) be the probability distribution of species in the initial 
generations, and the probability that individuals in the zth and jth species interbreed 
to produce an individual k. Then the probability distribution ) (the state) 

of the species in the first generation can be found by the total probability i.e. 



X l. 



^ Pij,k%i x ji k — l,...,J7l. 

*J=1 



This means that the association x° — > x' defines a map V called the evolution operator. 
The population evolves by starting from an arbitrary state x°, then passing to the state 
x' = V(x) (in the next "generation"), then to the state x" = V(V(x)), and so on. Thus 
states of the population described by the following dynamical system 

x°, x ' = V(x), x" = V 2 (x), x" = V 3 (x),... 



Note that V (defined by (1)) is a non linear (quadratic) operator, and it is higher di- 
mensional if to > 3. Higher dimensional dynamical systems are important but there are 
relatively few dynamical phenomena that are currently understood (pQ, [2], |18j). 

In this paper we consider a class of nonlinear (quadratic) operators which we call 
£-Volterra operators and study dynamical systems generated by such operators. 

The paper is organized as follows. 

In section 2 we give some preliminary definitions. Also we discuss the difference 
of quadratic operators introduced in this paper from known quadratic operators. In 
section 3 we describe some invariant (in particular some fixed points) sets for £-Volterra 
operators. Also we give some family of £-Volterra operators each element of which has 
cyclic orbits generated by several vertices of the simplex. We also show that the set of 
all ^-Volterra operators is convex, compact and describe its extremal points. Section 4 
devoted to 1-Volterra operators and section 5 devoted to 2-Volterra operators defined on 
a two dimensional simplex. In these sections we describe limit behavior of all trajectories 
(orbits). 



2 Preliminaries 

The quadratic stochastic operator (QSO) is a mapping of the simplex. 

S™- 1 = = ( Xl , ...,x m ) e R m : x t > 0, = lj (2) 

into itself , of the form 

m 

V ■ x k = ^ , Pij,k%i%ji k = 1, to, (3) 

where are coefficients of heredity and 

m 

Pijjk > 0, Pij,k = Pji,k, Pi i' k = l ' (*>^> k = 1 > m )' ( 4 ) 

k=l 

Thus each quadratic stochastic operator V can be uniquely defined by a cubic matrix 

P = i.Pij,k) n t ,j,k=i witn conditions (4). 

Note that each element x 6 S m ~ l is a probability distribution on E — {1, to}. The 
population evolves by starting from an arbitrary state (probability distribution on E) 
x G S 771 ' 1 then passing to the state V(x) (in the next "generation"), then to the state 
V(V(x)) = V 2 (x), and so on. 

For a given x^ E S m ~ l the trajectory (orbit) 

{ X W}, n = 0,1,2,... of x {0) 
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under the action of QSO (3) is denned by 



x (n+i) = v(x in) ), where n = 0,1,2, ... 

One of the main problem in mathematical biology consists in the study of the asymp- 
totical behavior of the trajectories. The difficulty of the problem depends on given matrix 
P. Now we shall briefly describe the history of (particularly) studied QSOs, which allows 
the reader to easily see the place of the operators introduced in this paper. 

The Volterra operators, (see j7], [9], [13]) A Volterra QSO is defined by (3), (4) and the 
additional assumption 

P ij)k = 0, if k#{i,j], Vi,j,keE. (5) 

The biological treatment of condition (5) is clear: The offspring repeats the genotype 
of one of its parents. 

In paper [7J the general form of Volterra QSO 

V : x = (xi, x m ) e S" 1 - 1 -> V(x) = x' = (x[, x'J G S^ 1 

is given 



Xu X k 



(6) 



i=i 



where 



Moreover 



a ki = 2P ikjk - 1 fori ^ k and a u = 0,i = 1, ...,m. 



aki = —<kk and \a ki \ < 1. 



In [7], [S] the theory of QSO (6) was developed by using theory of the Lyapunov 
function and tournaments. But non- Volterra QSOs (i.e. which do not satisfy the condition 
(5)) were not in completely studied. Because there is no any general theory which can be 
applied for investigation of non- Volterra operators. 

To the best of our knowledge, there are few papers devoted to such operators. Now 
we shall describe non- Volterra operators: 

A permutated Volterra operator. Papers [TU] , [12] are devoted to study of non- Voltera 
operators which are generated from Volterra operators (6) by a cyclic permutation of 
coordinates i.e 

K : = Xj M. + a ki x}j , i = 1, ...,m, 

where tc is a cyclic permutation on the set of indices E. 

Kvazi- Volterra QSO. In [1] a class of Kvazi- Volterra operators is introduced. For such 
operators the condition (5) is not satisfied only for very few values of i, j, k. 
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Non-Volterra QSO as combination of a Volterra and a non-Volterra operators. In 
paper [6] it was considered the following family of QSOs V\ : S 2 — » S 2 : 



V x = (l-X)V + XVi, 0< A< 1 



where 



V (x) = [x\ + 2xix 2 , x\ + 2x 2 x 3 , x\ + 2xix 3 ) 



is Volterra operator and 



Vx(x) = [x\ + 2X2^3, x 2 + 2XiX 3 , £3 + 2XiX 2 ) , 



is non-Volterra QSO. 

Note that behavior of the trajectories of Vq is very irregular (see [17], [22], |23j). 

Afon- Volterra QSO generated by a product measure. In [3] , [5] a constructive descrip- 
tion of the matrix P is given. This construction depends on a probability measure // 
which is given on a fixed graph G and cardinality of a set of cells (configurations). 

In [3] it was proven that the QSO constructed by the construction is Volterra iff G is 
a connected graph. 

In [19] using the construction of QSO for the general finite graph and probability 
measure \x (here /x is product of measures defined on maximal subgraphs of the graph G) 
a class of non-Volterra QSOs is described. 

It was shown that if fi is given by the product of the probability measures then corre- 
sponding non-Volterra operators can be studied by N number (where iV is the number of 
maximal connected subgraphs) of Volterra operators defined on the maximal connected 
subgraphs. 

F-QSO. Consider E = E{J{0} = {0, 1, ...,m}. Fix a set F C E and call this set the 
set of " females" and the set M = E \ F is called the set of " males" . The element will 
play the role of "empty-body". Coefficients P%^k °f the matrix P we define as follows 



Biological treatment of the coefficients (7) is very clear: a "child" k can be generated 
if its parents are taken from different classes F and M. 

For a given F C E a QSO with (3), (4) and (7) is called a F-QSO. Note that F-QSO 
is non-Volterra for any F C E. 

In [2n] the F-QSOs are studied for any F C E. It was proven that such operators has 
unique fixed point (1, 0, 0) G S m and all trajectories converge to this fixed point faster 
than any geometric progression. 




1, if k = 0,i,j G FU {0} or i,jeMU{0}; 
0, if k ^ 0,2, j G FU {0} or i,jeMU{0}; 
> 0, if ie F,j G M,V&. 



(7) 
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Strictly non-Volterra QSO. Recently in [21] a new class of non-Volterra operators is 
introduced. These operators satisfy 

Pi jt k = 0, if he {i,j}, Vi, j, k G E. (8) 

Such an operator is called strictly non-Volterra QSO. 

For arbitrary strictly non-Volterra QSO defined on S 2 in [21] it was proved that every 
such an operator has a unique fixed point. Also it was proven that, such operators have a 
cyclic trajectory. This is quite different behavior from the behavior of Volterra operators, 
since the Volterra operators have no any cyclic trajectory. 

Now we shall give a new class of non-Volterra operators. 

£- Volterra QSO. Fix £ G {1, ...,m} and assume that elements Pij± of the matrix P 
satisfy 

P ijlk = if k & {i,j} for any k G {1, ...,£}, i,j G E; (9) 

Pij,k > for at least one pair % ^ k, j ^ k if k G {£ + 1, ...,m}. (9a) 

Definition 1. For any fixed £ G {1, ...,m}, the QSO defined by (3), (4), (9) and (9a) 
is called £- Volterra QSO. 

Denote by Ve the set of all £- Volterra QSOs. 

Remarks. 1. The condition (9a) guarantees that Ve t f]Ve 2 =0 for any £\ ^ £ 2 . 

2. Note that £- Volterra QSO is Volterra if and only if £ = m. 

3. Kvazi- Volterra operators (introduce above) are particular case of £- Volterra opera- 
tors. 

4. The class of £- Volterra QSO for a given £ does not coincide with a class of non- 
Volterra QSOs mentioned above. 

We shall use the following notions. 

Definition 2.([T], p. 215 )A fixed point P for F : R m — > R m is called hyperbolic if 
the Jacobian matrix J of the map F at the point P has no eigenvalues on the unit circle. 
There are three types of hyperbolic fixed points : 

1. P is an attracting fixed point if all of the eigenvalues of 3{P) are less than one in 
absolute value. 

2. P is a repelling fixed point if all of the eigenvalues of J{P) are greater than one in 
absolute value. 

3. P is a saddle point otherwise. 

The following theorem is also very useful. 

Theorem 1. ([I], p. 217) Suppose F has a saddle fixed point P. There exist e > 
and a smooth curve 7 : (— s, e) —* R 2 such that 7(0) = P; j'(t) 7^ 0; 7'(0) is an unstable 
eigenvector for J(P); 7 is F^ 1 — invariant; F~ n (7(t)) — > P as n — > 00; if\F~ n (q) — P\ < e 
for all n > then q = ^(t) for some t. 
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The curve 7 is called the (local) unstable manifold at P. The theorem is true for stable 
sets as well as with the obvious modification. On the local manifold, all points tend to 
the fixed point under iteration of F. 



3 Canonical form of ^-Volterra QSO. 



By definition for k — 1, £ we have 



x 



k ^ ^ Pij,k%i%j Pkk,k%k 2 ^ ^ Pik,k%i%k 



( 



\ 



Pkk,kXk + 2 Pi 



ik,k%i 



\ 

Using x k = 1 — x i we § e t 



/ 



\ 



Pkk,k + ^ (2Fjfc,fc — -Pfcfc,fc) 



V 



, A; — 1, £. 



J 



For k — t + 1, m we have 

( 



Pkk,k ~\~ ^ ^ (2>Pjk,k Pkk,k) x i + ^ ^ P'lj,k x i x j- 



\ 



/»>J=1 



Denote a fci = 2P ifcifc - P fcfc)fc then 

= %k [ &kk + X/ i=1 a ki%i ) , A; = 1, £ 
x ' k = Xk \ Ctkk 4" J^ i=1 &ki%i ) "I" X/ Pij,k%i%ji k — £ + 1, 

Note that < a^k < 1 and — < < 2 — a^, i ^ k, < Pij^k < 1- 
For any / C E — {1, 2, m} we define the face of the simplex S 1 " 1-1 : 

r,= {iG S™" 1 : rri = for any i e 1} . 



m. 



(10) 



Proposition 1. Let V be a l-Volterra QSO. Then the following are true 

(i) Any face Tj with I C {1, ...,£} is invariant set with respect to V. 

(ii) Let A t — {i e {!,...,£} : an > 0}. For any I C A e II {£ + l,...,m} £/ie se£ 
Tj = {x e S""^ 1 : rcj > 0, Vi G /} is invariant with respect to V. 
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Proof, (i) From (10) it follows that if Xi = then x\ — for any % G {1, ...,£}. Hence 

v(r f )crj if/c{i,...,0- 

(ii) Take / C A^U{£+1, ...,m}. For k G IC\A(, by (10) and inequality — Okk < Ofej, j = 
1, m we get 

/ m \ 



X k ~ X k 



Clkk 



> 



( 



Xk 



a kk — a kk X 



\ 



'3 



dkk x k > 0, since Xk > for k e I H A e . 



J 



For fc G / Pi {£+ 1,..., m} by (10) and condition (9a) we have 

^ m, \ m 



X k ~ X k 



akk 



+ ^ ] CLkjXj 



\ 



3 = 1 

j^k 



-\- ^ ^ Pij kXiXj ^ 



7 i^fc 



/ 



Xk 



\ 



\ 



3 = 1 



akk - a k k + Y P i3\kXi x j > dkkx\ > 0) 



/ 



i,3£I 
i^k 
3^k 



here we used Yl v e/ Pij,k%iXj > which follows from condition (9a) and Xi > 0,Xj > 
0, G /. Thus V(7» C Tj if J C ^ U {t + 1, ...,m}. The proposition is proved. 

Denote e« = ...,5 mi ) G <S m_1 , i = 1, ...,m the vertices of the simplex S*" 1-1 , where 
5ij is the Kronecker's symbol. 

Proposition 2. 1) The vertex t{ is a fixed point for a l-Volterra QSO iff Pa,i = 1, 
(i = 1, ...,m). 

2) For any collection I s = {e^, e is } C {e^ +1 , e m }, (s < m — £) there exists a 
family Vi(I s ) C Vt such that {e^, ei a } is a s-cycle for each V G Vz(I s )- 

Proof. 1) It is easy to see that if i G {1, .-.,1} then 

m 

V{e,j) = (0, 0, P iiti , 0, 0, Pu,t+u Pu,rn) with P iM + ^ p a,3 = 1 
and if % G {£ + 1, m} then 



V(ei) = (0, 0, Pii, m ) with = 1. 

j=£+l 



(11) 



Thus V(e i ) = e i iff P iM = 1. 
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2) By (11) we have 
for any j — 1, s. In order to get V^e^) = e i2 we assume 

Piiii,i 2 = 1) Piihj = 0; J 7^ *2- (Hi) 

Then to get V(e i2 ) = e ia we assume 

PiiiiM = 1) -Pj2i2,j = 0; i 7^ *3- (H2) 

Similarly to get V(ej s _ 1 ) = e ia we assume 

Pi 3 -ii 3 -i,i s = 1) Pi 3 -ii 3 -i,j = 0, j 7^ i s - (ll s _i) 

The last assumption follows from V(e ia ) = e h i.e 

Pi s i s ,h = 1, = 0, j ^ »i- (H a ) 

Hence Vg(/ S ) = {V G Vt : the coefficients of V satisfy (Hi) — (ll s )}. The proposition 
is proved. 

For any set A denote by |A| its cardinality. 

The next proposition gives a set of periodic orbits of £- Volterra QSOs. 

Proposition 3. For any I 1: I q C {£ + 1, m} such that I { f] Ij = (i ^ j, i, j = 

1, q). There exists a family Ve(h, Iq) C Vi such that each collection {ei,i G Ij}, j = 
1, q is a cycle for every V G Vg(Ii, I q ). 

Proof. Since Ii — 0, % ^ j the family can be constructed using Proposition 2 i.e. 

v<(/i,...,/ 9 ) = ns=iW). 

Remarks. 1) There is not any Volterra operator with a periodic orbit {e ii; e is } C 
{ei, e e }, 1 < s < t. 

2) Propositions 2 and 3 show that £- Volterra operators have quite different behavior 
from the behavior of Volterra operators, since Volterra operators have no cyclic trajecto- 
ries. 

Recall that Vi is the set of all £- Volterra operators defined on S m ~ l . 

m(m-l)(m-l+l) 

Proposition 4. (i) The set Vi is a convex, compact subset of R 2 

(ii) The extremal points ofVe are l-Volterra operators with Pij t k = or 1 for any i,j, k 

i.e. 

Extr(V£) = {V G Ve : the matrix P of V contains only and 1} . 

(iii) If£ = m then |Extr(V*)| = 2^ m( - m ~ 1 \- if £ < m - 1 then 

|Extr(V,)| = (m - £f m - mm '" +1) (m - £ + l) (m " m)/ (m - ^ + 2)^" 1) . 
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Proof, (i) Since we have one-to-one correspondence between the set of all QSOs and 
the set of all cubic matrices P, we can consider a QSO V as a point of R/»(™ 2 -i). The 
number m ( m - 1 )^ a - e + 1 ) [ s the number of independent elements of the matrix P with the 
condition (9). Let V\,V 2 be two £-Volterra QSO i.e V\,V 2 G Ve. We shall prove that 
V = \V l + (l- X)V 2 G V e for any A G [0, 1]. 

Let P^j, (resp. Pjj^) be coefficients of V\ (resp. V 2 ). Then coefficients of V has the 
form 

P** = + (1 - A)P$. (12) 

By definition coefficients P^ k and P^ k satisfy conditions (9) and (9a). Using (12) it is 
easy to check that P^ also satisfy the condition (9) and (9a). 

(ii) Assume V G Ve with Pi j 0t k = « 7^ and 1 for some io,jo,k . Construct two 
operators V q with coefficients pM, q — 1,2 as following 




Pij,k if (^0, jo), 

1 if (hj,k) = (io,jo,ko), 

if (ij,k) = (io,j ,h), k ^ k 



{Pij,k if ^ (io,jo), 

if k) = (i ,jo,ko), 
^ if (i,j,k) = (i ,j ,k), k^k . 

Then 

{Pij,k if ^ (io, jo), 

a = P wo , fco if (i, j, k) = (i ,j , k ) = P lhk . (12') 
Pij,k if k) = (i ,jo, k), k^k 

Since a > 0, from (12') we get P^ = if and only if P^ = and P-jj, = 0. This means 
that V\ and V2 are ^-Volterra operators. Hence V = aV\ + (1 — a) V 2 . Thus if P^ G (0, 1) 
for some k) then V is not an extremal point. Finally, if P^ = or 1 for any k) 
then the representation V = XV 1 + (1 — \)V 2 , < A < 1 is possible only if Vi = V 2 = V. 

(iii) In order to compute cardinality of Extr(V^) we have to know which elements of 
the matrix P can be 1. 

Denote Vij = (Pij,i, Pij,mf the (i,j)ih column of P, where G fC = : 1 < 

i < j < m}. 

Let n (Vij) be the number of elements of Vij which must be zero by conditions (4), 
(9), (9a). 

Put for £ G {1, ...,m} : 

A = A em = G K : % < £, j G {0 U {£ + 1, m}} , 

B = B em = G K : i < £,j < £,i < j}, 
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C = C em = {(i,j)elC:e<i<j}. 

Note that IC = AUBUC. If £ = m then C = 0. 
It is easy to see that 

( £-1 if (i,j)eA 
1 € if («,j)GC 

By condition (4) each column contains unique "1". We have m — no(Vij) possibilities 
to write 1 in the column € /C. Thus 

|Extr(V*)| = n {m-n {V tj )) = {m-£+l) lAl {m-£ + 2) lBl {m-£f l . 
This with 

\A\ = (m-e + i)e, \B\ = he-i)e, \c\ = hm-e + i)(m-e), 

would yield the formula. The proposition is proved. 

mirrP' — 1) 

For the set V of all QSOs we have V C R 5 . Note that V also is a convex, 
compact set. Its extremal points also are operators with = or 1 only. It is easy to 
see that 

|Extr(V m )| < |Extr(V m _i)| < ... < |Extr(Vi)| < |Extr(V)| = m ^ m{m+1) . 
For example, if m — 3 then 

|Extr(V 3 )| = 8, |Extr(V 2 )| = 48, |Extr(Vi)| = 216, |Extr(V)| = 729. 

The set V can be written as V = \J™ =0 V e . Here V is the set of " O-Volterra QSO"s 
i.e for any k G {1, ...,m} there is at least one pair with i ^ k and j ^ k such that 

Pij,k > 0. 

As it was mentioned above: V m is the set of all Volterra operators and Ve 1 f)Ve 2 = ® 
for any £\ ^ £ 2 G {0, ...,m}. 

Thus to study dynamics of QSOs from V it is enough to study the problem for each 
V e ,£ = 0,...,m. 

In general, the problem of study the behavior of V C Vg (for fixed £) is also a difficult 
problem. So in the next sections we consider the problem for small dimensions (i.e m = 
2,3) and£= 1,2. 



4 Case m = 2 

In the case m = 2 we have only 1- Volterra operator V : S 1 — > S" 1 such that 

x' = ax 2 + 2cxy 
y' = bz 2 + 2rfxy + y 2 , 

10 



where a, b,c,d G [0, 1) (the case a = 1 corresponds to Volterra operator), a + b = c+d = 1. 
Using a; + y = 1 from (13) we get a dynamical system generated by function f(x) = 
(a — 2c)x 2 + 2cx, x G [0, 1], a G [0, 1), c G [0, 1]. By properties of f(x) one can prove the 
following 

Proposition 5. 1) // c < | ? Va G [0,1) £/ie operator (13) has unique fixed point 
A = (0, 1) and for any initial point A = (x°,y°) G S" 1 the trajectory A*™- 1 goes to A as 
n — > oo. 

2) 7/c > |, Va G [0, 1) inen nas two fixed points A = (0,1) and A* = (g^, ^) 
ine pom£ Ao zs repeller. For any initial point A G S 1 \ {Ao} the trajectory A*™) tends to 
A* as n — > oo. 

5 Case m = 3 

In case m = 3 one has two ^- Volterra operators (for £ = 2 and 1). Here we shall study 
the 2- Volterra operators. 

Arbitrary 2- Volterra operator (for m — 3) has the form : 

x' = x{a\x + 2b\y + 2c\z) 

y' = y{2b 2 x + d lV + 2e x z) (14) 
z' = z(2c 2 x + 2e 2 y + z) + a 2 x 2 + 2b 3 xy + d 2 y 2 , 

where 

Qi — Pn,i, 0-2 — Pii,3j h — Pi2,i,i — 1)2,3; C\ = P^i, 

c 2 = P13& d i = P 22:i , i = 2, 3; e { = P 23 ,i, i = 2, 3. (15) 
To avoid many special cases and complicated formulas we consider the case 

Pl,l = -P22,2, Pl3,l — ^23,2, Pl.2,1 = Pl.2,2- (16) 

This corresponds to a symmetric (with respect to permutations of 1 and 2) model. 
Using x + y + z = 1 and condition (16) the operator (14) can be written as 

f x' = x(2c + (a - 2c)x + 2(6 - c)y) 
\y' = y(2c + 2(6 - c)x + (a - 2c)y), 

where a = P nA G [0, 1), b = P 12 ,i G [0, |], c = Pi 3) i G [0, 1], and x, y G [0, 1] such that 
x + y < 1. 

Remark. The case a = Pi^i = P 22j2 = 1 corresponds to the Volterra case, so we 
consider only a ^ 1. 

Theorem 2. (i) For c < \ the operator (17) has unique fixed point X = (0,0) which 
is global attractive point. 
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(ii) Sets M = {\= (x,y) : x = 0}, M 1 = {A = (x,y) : y = 0}, M= = {A = (x,y) : 
x = y}, M > = {A = (x, y) : x > y} ; M< = {A = (x, y) : x < y} are invariant with respect 
to the operator (17). 

(iii) For c > \, a ^ 2b the operator (17) has four fixed points X = (0,0), Ai = 
(°> M)> A 2 = (1S,0) ; A 3 = (^, ^^). Moreover A is repeller and 



Ai and A2 are 



attractive, if a > 2b 
saddle, if a < 2b 



A3 «s 



attractive, if a < 2b 
saddle, if a > 2b. 



(iv) For c > I , a = 2b the operator (17) has a repeller fixed point X = (0,0) and 
continuum set of fixed points F = {A = (x, y) : x + y = 2 ^-b) }• The following ^ ne 

h = {A = (x, y) : y = vx, x £ [0, 1]} 



is an invariant set for any v G [0, 00). // A = (x°, y°) is an initial point with ^ — u, 
(x° ^ 0) then its trajectory A (n) goes to \ u = ( 2 (c-b)(i+v) > 2(¥-b)(i+v) ) G ^ H F as n ~^ 00 > 
v E [0, 00), (if x° = tnen on invariant set Mq we have A^ — > A]J. 

(v) If a < 2b then M (resp. Mi) is the stable manifold of the saddle point Ai (resp. 
A2). If a > 2b then M = is the stable manifold of saddle point A3. There is an invariant 
curve 7 passing through Ai, A2, A3 which is unstable manifold for the saddle points. 

Proof, (i) Clearly A = (0, 0) is a fixed point for (17). Note that the Jacobian of (17) 
at (0,0) has the form 

' 2c 



J 



2c 



so Ao is an attractive if c < \ and non- hyperbolic if c = \. 

Now we shall prove (for c < |) its global attractiveness. From the first equation of 
(17) we have 

x = x(ax + 2by + 2cz) < qx, (18) 

where q = max{a, 26, 2c}. By definition of the operator (17) and condition c < | we have 
q < 1. Consider two cases: 

Case g < 1. In this case from (18) we get x n+1 < qx n < q n x°, where x n is the first 
coordinate of the trajectory A^ = V n (l°) = (x n , y n ) with initial point A = (x°, y°). Thus 
x n — > as n — > 00. By symmetry of x and y we get y n — > as n — > 00. 

Case g = 1. In this case we get x n+ i < x n , hence 

lim x n = a > exists. 
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Similarly, 

lim y n = (3 also exists. 

Thus the point (a, (3) must be a fixed point for the operator (17). Since A = (0,0) is 
unique fixed point for c < \ (we shall prove uniqueness in section (iii) of this proof), we 
get (a, (3) = (0,0). 

Remark. The argument used in the case q = 1 also works for the case q < 1. But in 
the case q < 1 we proved that the rate of convergence to Ao is faster than q n . 

(ii) Invariance of Mq, M=, Mi are straightforward. Invariance of M<, M> follow from 
the following equality 

x' — y' — (x — y)(2cz + a(x + y)), where z — 1 — x — y > 

which can be obtained from (17). 

(iii) Clearly A = (0, 0) is a fixed point independently on parameters a, b, c. To get 
other fixed points consider several cases: 

Case x = 0, y ^ 0: From the second equation one gets y = |^ which is between 
and 1 iff c > \. Thus Ai = (0, is a fixed point. 

Case x 7^ 0, y = is similar to the previous case and gives A 2 = (f^, 0)- 

Case x 7^ 0, y ^ 0: From (17) one gets a system of linear equations, which has unique 
solution A 3 = ( 4^^-26 ' Ac-a-2b ) c> |> a 7^ 26). Note that if c < \ then there is only 
A . 

To check the type of fixed points consider Jacobian at A = (x, y) 

I 2c + 2(a-2c)x + 2(b- c)y 2(b - c)x 

J(A)=J(x,i/)= | ■ (19) 

y 2(b-c)y 2c + 2(a-2c)y + 2(b-c)x 

It is easy to see that the eigenvalues /ii(A), /i 2 (A) of (19) at fixed points are 

Ati(A 2 ) = A* 2 (Ai) = Ai 2 (A 3 ) = 2(1 - c) < 1, 



l^i( A i)l = l/^2(A 2 )| 



2c(l - a) + 2b(c - 1) 



Ia*i(a 3 



2c — a 
Ac(b - 1) + 2a(c - 1) 



< 1 if a > 2b 
> 1 if a < 2b 



a + 2b - 4c 



< 1 if a < 2b 
> 1 if a > 2b 



This completes the proof of (iii). 

(iv) For a = 2b the operator (17) has the following form 

x' = x(2c + 2(b- c)(x + y)) 
y> = y{2c+2{b-c){x + y)). 
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(20) 




Lp'(x*) = 0. The graphical analysis shows that x is the global attractive point on I v . 

(v) The existence of 7 follows from Theorem 1. Other statements of (v) are straight- 
forward. The theorem is proved. 

Note that 2-Volterra operator corresponding to (17) has the following form 



Using Theorem 2 one can describe the phase portraits of the trajectories of (21). 

Remark. One of the main goal by introducing the notion of Volterra operators was 
to give an example of QSO which has more rich dynamics than Volterra QSO. It is well 
known [7] that for Volterra operators (see (6)) if 7^ (i ^ j) then for any non- fixed 
initial point A the set u(X°) of all limit points of the trajectory {A^} is subset of the 
boundary of simplex. But in our case Theorem 2 shows that the limit set need not to be 
subset of the boundary of S 2 . 
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